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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 
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Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 
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1)^ Responsive to communication(s) filed on 18 January 2005 . 
2a)D This action is FINAL. 2b)E This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 
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5) D Claim(s) is/are allowed. 

6) S Claim(s) 1-30. 58-60 is/are rejected. 
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Application Papers 

9) D The specification is objected to by the Examiner. 

10)13 The drawing(s) filed on 03 December 2001 is/are: a)D accepted or b)E3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
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DETAILED ACTION 

Election/Restrictions 

1 . Applicant's election without traverse of Group I (claims 1-30 and 58-60) in the reply filed on 
January 18, 2005 is acknowledged. 

2. Please cancel the non-elected claims in case of allowability. 

Information Disclosure Statement 

3. The information disclosure statements (IDS) submitted on August 29, 2003 and September 22, 
2003 have been considered by the examiner. 

Drawings 

4. The drawings are objected to because the box elements (such as 272, 274, etc.) in Figure 6 need 
to be labeled in accordance with 37 C.F.R. § 1.83(a) as stated infra. A proposed drawing correction or 
corrected drawings are required in reply to the Office action to avoid abandonment of the application. 
The objection to the drawings will not be held in abeyance. 

Specifically, 37 C.F.R. § 1.83(a) states that "the drawing in a nonprovisional application must 
show every feature of the invention specified in the claims. However, conventional features disclosed in 
the description and claims, where their detailed illustration is not essential for a proper understanding of 
the invention, should be illustrated in the drawing in the form of a graphical drawing symbol or a labeled 
representation (e.g., a labeled rectangular box). 5 ' 

Claim Objections 

5. Claim 60 is objected to because of the following informalities: claim 60 line 2 discloses 
"wherein" after "predicting". Perhaps the applicant meant to say "where" instead of "wherein"? 
Appropriate correction is required. 

Claim Rejections - 35 USC § 112 

6. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 
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The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

7. Claims 9-10 and 19-26 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant regards as the 
invention. 

8. Claim 9 recites the limitation "the selected segments" in line 3 and "the portion" in line 5 on page 
60. There is insufficient antecedent basis for this limitation in the claim. 

9. Claim 19 recites the limitation "the location of the face" in lines 7-8 on page 62. There is 
insufficient antecedent basis for this limitation in the claim. 

10. Claim 26 recites the limitation "the location of the face" in lines 3-4 on page 64. There is 
insufficient antecedent basis for this limitation in the claim. 

1 1 . Claims 10 and 20-25 are rejected for the same reasons as claims 9 and 19. 

Claim Rejections - 35 USC § 102 

12. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis 
for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

13. Claims 1-7, 13-16 and 58 are rejected under 35 U.S.C. 102(e) as being anticipated by de Cuetos 
et al. (hereinafter, "de Cuetos 55 ) (US 6,754,373). 

With regard to claim 1 de Cuetos discloses a method comprising: receiving a frame of content 
(col. 3 lines 34-35 and 41-49), automatically detecting a candidate area for a new face region in the frame 
(image feature tracker to detect a face of an user, col. 3 lines 50-67, col. 4 lines 1-7); using one or more 
hierarchical (de Cuetos defines this as a class levels) verification levels to verify whether a human face is 
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in the candidate area (depending on the region of interest (ROI) such as mouth, tongue, teeth, etc. the 
levels (i.e., class: speech and/or non-speech segments) are defined by the classifier 32 at col. 5 lines 14- 
16); indicating that the candidate area includes a face if the one or more hierarchical verification levels 
verify that a human face is in the candidate area (col. 5 line 60 to col. 6 lines 1-4); and using a plurality of 
cues (visual speech cues at col. 3 lines 14-22) to track each verified face in the content from frame to 
frame (col. 3 lines 50-67 and col. 5 lines 33-52). The examiner understands that de Cuetos teaches visual 
speech cues. However, de Cuetos teaches analyzing VISUAL speech cues and tracking these cues 
(obviously from a human face) from the each image that is being captured by the camera system 12. 

With regard to claims 2-4 de Cuetos discloses both audio and video content as seen at elements 
18 and 26 in Figure 1 and at col.4 lines 1-20. 

With regard to claim 5 de Cuetos discloses repeating automatic detecting in the even tracking of a 
verified face is lost as seen in Figure 2 as seen from step 120 back to step 1 12. 

With regard to claim 6 de Cuetos discloses receiving the frame of content comprises receiving a 
frame of video content from a video capture device local to a system implementing the method (as seen in 
Figure 1, the system of local). 

With regard to claim 7 de Cuetos discloses receiving the frame of content comprises receiving 
the frame of content from a computer readable medium accessible to a system implementing the method 
(computer system disclosed in Figure 1). 

With regard to claim 13 de Cuetos discloses the candidate area including a face comprising: 
recording the candidate area in a tracking list (see Face Tracker and Feature Tracker 18). 

With regard to claim 14 de Cuetos discloses recording the candidate area in the tracking list 
comprising accessing a record corresponding to the candidate area and resetting a time since last 
verification of the candidate (see Countdown timed out? Element 122 in Figure 2). 
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With regard to claim 15 de Cuetos discloses the one or more hierarchical verification levels 
include a first level and a second level, and wherein using the one or more hierarchical verification levels 
to verify whether the human face is in the candidate area comprises: checking whether, using the first 
level verification, the human face is verified as in the candidate area; and using the second level 
verification only if the checking indicates that the human face is not verified as in the candidate area by 
the first level verification (de Cuetos uses the classifier 32 to classify features at different level. Verifying 
continuously whether the person is talking or not and whether the person comes in front of a camera. See, 
col. 5 line 32 to col. 6 lines 1-30). 

Claim 16 recites similar features as claim 15. Thus, arguments similar to that presented above 
for claim 15 is equally applicable to claim 16. Note that de Cuetos discloses identifying human face from 
the ROI mentioned above. 

Claim 58 recites identical features as claim 1. Thus, arguments similar to that presented above 
for claim 1 is equally applicable to claim 58. See claim 1 where de Cuetos discloses plurality of cues to 
track an object. 

Claim Rejections - 35 USC § 103 

14. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 

rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

15. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over de Cuetos et al. 
(hereinafter, "de Cuetos") (US 6,754,373) in view of Kellner (US 6,539,099). 

With regard to claim 8 de Cuetos discloses a method as disclosed above in claim 1 and the 
arguments are not repeated herein, but are incorporated by reference, de Cuetos discloses detecting 
whether there is motion in the frame and, if there is motion in the frame, then performing motion-based 
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initialization to identify one or more candidate areas (col. 3 lines 50-67); detecting whether there is audio 
in the frame, and if there is audio in the frame, then performing audio-based initialization to identify one 
or more candidate areas (col. 4 lines 1-50). de Cuetos does not expressly disclose using, if there is neither 
motion nor audio in the frame, a fast face detector to identify one or more candidate areas. Kellner 
discloses face detector at col. 4 lines 45-52. At the time of the invention, it would have been obvious to a 
person of ordinary skill in the art to combine the teaching of Kellner with de Cuetos. The motivation for 
doing so is to allow more individuals to participate in the same video session as suggested by Kellner at 
col. 3 lines 1-12. Therefore, it would have been obvious to combine Kellner with de Cuetos to obtain the 
invention as specified in claim 8. 

16. Claim 1 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over de Cuetos et al. 

A 

(hereinafter, "de Cuetos") (US 6,754,373). 

With regard to claim 11 it would have been obvious matter of design choice to modify the de 
Cuetos reference by having levels in the hierarchical verifications levels to include a level to verify a ROI 
faster but less accuracy than the other level since applicant has not discloses that having this coarse levels 
solves any stated problem or is for any particular purpose and it appears that the levels defied in claim 1 1 
would perform equally well with the levels defined as in claim 1 (as disclosed in de Cuetos). 

Claim 17 recites identical features as claim 1 1 . Thus, arguments similar to that presented above 
for claim 1 1 is equally applicable to claim 17. 

17. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over de Cuetos et al. 
(hereinafter, "de Cuetos") (US 6,754,373) in view of Wang et al. (hereinafter, "Wang") (US 5,805,733). 

With regard to claim 12 de Cuetos discloses the method as disclosed above in claim 1 and the 
arguments are not repeated herein, but are incorporated by reference, de Cuetos does not expressly 
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disclose generating a color histogram of the candidate area; generating an estimated color histogram of 
the candidate area based on previous frames; determining a similarity value between the color histogram 
and the estimated color histogram; and verifying that the candidate area includes a face if the similarity 
value is greater than a threshold value. Wang discloses this at col 4 lines 9-35. At the time of the 
invention, it would have been obvious to a person of ordinary skill in the art to combine the teaching of 
Wang with de Cuetos. The motivation for doing so is for computation efficiency, by reducing the search 
space, because all scenes in a group of related scenes need not appear in the window at the same time as 
suggested by Wang at col. 2 lines 1-15. Therefore, it would have been obvious to combine Wang with de 
Cuetos to obtain the invention as specified in claim 12. 

18. Claims 18, 27-28 and 59-60 are rejected under 35 U.S.C. 103(a) as being unpatentable over de 
Cuetos et al. (hereinafter, "de Cuetos") (US 6,754,373) in view of Murakami et al. (hereinafter, 
"Murakami' 5 ) (US 6,798,834). 

With regard to claim 27 de Cuetos discloses the method as disclosed above in claim 1 and the 
arguments are not repeated herein, but are incorporated by reference, de Cuetos does not expressly 
disclose using plurality of cues to track each verified face comprising concurrently tracking multiple 
possible locations for the face from frame to frame. Murakami discloses tracking multiple possible 
locations at col. 6 lines 48-64. At the time of the invention, it would have been obvious to a person of 
ordinary skill in the art to combine the teaching of Murakami with de Cuetos. The motivation for doing 
so is to provide a motion prediction circuit, which can correctly perform motion prediction and improve 
the efficiency of coding by coding an image as suggested by Murakami at col. 3 lines 20-3 1 . Therefore, it 
would have been obvious to combine Murakami with de Cuetos to obtain the invention as specified in 
claim 27. 

With regard to claim 28 Murakami discloses multiple hypothesis tracking technique to track the 
multiple locations such as a moving person 201, motionless person 202, a desk 203, etc. 
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With regard to claim 18 Both de Cuetos and Murakami discloses plurality of cues as color 
(Murakami, col. 16 lines 6-8), edge, motion, and audio (de Cuetos, col. 3 lines 50 to col. 4 lines 1-7). 

Claim 59 recites identical features as claim 18. Thus, arguments similar to that presented above 
for claim 18 is equally applicable to claim 59. 

With regard to claim 60 de Cuetos discloses a method as disclosed above in claim 1 and the 
arguments are not repeated herein, but are incorporated by reference, de Cuetos does not expressly 
disclose predicting where the object will be from frame to frame based on the plurality of cues. However, 
this concept is conventional in the art. Murakami discloses this at col. 15 lines 55 to col. 16 lines 1-7. At 
the time of the invention, it would have been obvious to a person of ordinary skill in the art to combine 
the teaching of Murakami with de Cuetos. The motivation for doing so is to predict the object as 
disclosed by Murakami. Therefore, it would have been obvious to combine Murakami with de Cuetos to 
obtain the invention as specified in claim 60. 

19. Claims 29-30 are rejected under 35 U.S.C. 103(a) as being unpatentable over de Cuetos et al. 
(hereinafter, "de Cuetos") (US 6,754,373) in view of Murakami et al. (hereinafter, "Murakami") (US 
6,798,834) as applied to claims 27-28 above, and further in view of Freeman et al. (hereinafter, 
"Freeman") (US 6,766,042). 

With regard to claim 29-30 de Cuetos (as modified by Murakami) discloses the method as 
disclosed above in claims 1 and 27-28 and the arguments are not repeated herein, but are incorporated by 
reference. Neither de Cuetos nor Murakami expressly disclose particle filter. Freeman discloses particle 
filtering at col. 8 lines 41-61 . At the time of the invention, it would have been obvious to a person of 
ordinary skill in the art to combine the teaching of Freeman with Murakami and de Cuetos. The 
motivation for doing so is to determine a set of parametric values from the image as suggested by 
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Freeman. Therefore, it would have been obvious to combine Freeman with Murakami and de Cuetos to 
obtain the invention as specified in claims 29-30. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the examiner should 
be directed to Shefali D. Patel whose telephone number is 571-272-7396. The examiner can normally be 
reached on M-F 8:00am - 5:00pm (First Friday Off). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Joseph Mancuso can be reached on (571) 272-7695. The fax phone number for the organization where 
this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be obtained 
from either Private PAIR or Public PAIR. Status information for unpublished applications is available 
through Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

Shefali D Patel 
Examiner 
Art Unit 2621 

August 18, 2005 // 



